The following analysis is built upon the International Energy Agency’s (IEA) IEA Headline Energy Data, which includes product data for 14 energy balance flows in kilotons of oil equivalent (ktoe) for 9 energy products over the years 1971-2014.
The IEA is an independent agency that works closely with the Organisation for Economic Co-operation and Development (OECD), an intergovernmental economic organisation with 35 member countries, founded in 1961 to stimulate economic progress and world trade (Wikipedia “OECD” 11/15/16).
This dataset can be downloaded from the IEA’s website: https://www.iea.org/statistics/
head(df,2)
## Country Product Flow X1971 X1972 X1973
## 1 Australia Coal, peat and oil shale Production (ktoe) 32680 39372 40251
## 2 Australia Coal, peat and oil shale Imports (ktoe) 0 0 0
## X1974 X1975 X1976 X1977 X1978 X1979 X1980 X1981 X1982 X1983 X1984 X1985
## 1 39179 44767 44161 48910 47947 51970 51898 62618 63335 72458 78958 83939
## 2 0 0 0 0 7 9 1 0 1 0 0 0
## X1986 X1987 X1988 X1989 X1990 X1991 X1992 X1993 X1994 X1995
## 1 90500 102763 90074 100214 106103 114763 119139 119983 118687 127823
## 2 0 0 0 0 0 0 0 0 0 0
## X1996 X1997 X1998 X1999 X2000 X2001 X2002 X2003 X2004 X2005
## 1 129956 140676 152427 154539 164578 178433 183863 183860 189189 201579
## 2 0 0 0 0 0 0 0 35 10 14
## X2006 X2007 X2008 X2009 X2010 X2011 X2012 X2013 X2014
## 1 203808 214979 215910 223333 246564 233641 245351 264517 285440
## 2 33 28 5 4 43 161 87 62 126
names(df)
## [1] "Country" "Product" "Flow" "X1971" "X1972" "X1973" "X1974"
## [8] "X1975" "X1976" "X1977" "X1978" "X1979" "X1980" "X1981"
## [15] "X1982" "X1983" "X1984" "X1985" "X1986" "X1987" "X1988"
## [22] "X1989" "X1990" "X1991" "X1992" "X1993" "X1994" "X1995"
## [29] "X1996" "X1997" "X1998" "X1999" "X2000" "X2001" "X2002"
## [36] "X2003" "X2004" "X2005" "X2006" "X2007" "X2008" "X2009"
## [43] "X2010" "X2011" "X2012" "X2013" "X2014"
unique(df$Country)
## [1] Australia Austria
## [3] Belgium Canada
## [5] Chile Czech Republic
## [7] Denmark Estonia
## [9] Finland France
## [11] Germany Greece
## [13] Hungary Iceland
## [15] Ireland Israel
## [17] Italy Japan
## [19] Korea Luxembourg
## [21] Mexico Netherlands
## [23] New Zealand Norway
## [25] Poland Portugal
## [27] Slovak Republic Slovenia
## [29] Spain Sweden
## [31] Switzerland Turkey
## [33] United Kingdom United States
## [35] OECD Total People's Republic of China
## [37] Indonesia Thailand
## [39] Non-OECD Total Africa
## [41] Non-OECD Americas Asia (including China)
## [43] Non-OECD Europe and Eurasia Middle East
## [45] World
## 45 Levels: Africa Asia (including China) Australia Austria ... World
unique(df$Product)
## [1] Coal, peat and oil shale Crude,NGL and feedstocks
## [3] Oil products Natural gas
## [5] Nuclear Renewables and waste
## [7] Electricity Heat
## [9] Total Fossil Fuels
## [11] Renewable sources
## 11 Levels: Coal, peat and oil shale ... Total
unique(df$Flow)
## [1] Production (ktoe)
## [2] Imports (ktoe)
## [3] Exports (ktoe)
## [4] Total primary energy supply (ktoe)
## [5] Electricity plants (ktoe)
## [6] CHP plants (ktoe)
## [7] Heat plants (ktoe)
## [8] Oil refineries, transformation (ktoe)
## [9] Total final consumption (ktoe)
## [10] Industry (ktoe)
## [11] Transport (ktoe)
## [12] Other (ktoe)
## [13] of which: Residential (ktoe)
## [14] of which: Commerce and public services (ktoe)
## [15] Electricity output (GWh)
## [16] Total CO2 emissions - Fuel Combustion (Mt of CO2)
## 16 Levels: CHP plants (ktoe) ... Transport (ktoe)
This datset contains 52 variables with 5,895 observations, with annual data encompassing 1971 to 2014 (5 variables were excluded because they were empty; these were projected placeholder variables for 2015 total and quarterly data). The data are condusive for within-country comparisons between energy products and between-country comparisions of specific energy products. In addition to country data, world totals and regional data are also provided.
To capture some of the breadth of these data, I have organized this report by analyses pertainting to USA, World and Regions.
The dataset contains 16 levels of energy flow categories, and I chose to focus my analysis on imports and exports. To get a snapshot of US energy imports, I plotted a pie chart of 2014 product percentages. To do so, I excluded two products that had no data (Heat and Nuclear), and combined the three smallest imported products (Crude, Renewables, and Electricity) for greater legilibility.
Looking just at this 2014 snapshot, Crude makes up a whopping 75% of US energy imports. The supporting documentation for this dataset defined crude as crude oil, natural gas liquids, refinery feedstocks, and hydrocarbons. I wanted to see if these energy product proportions persisted across the entire timeframe of the dataset, so I plotted the longitduinal data of each energy product import together.
Crude imports surpass the next leading energy import by a large margin, and seem worthy of further investigation. Before analyzing them further, I wanted to see the dynamics of the other energy products in a plot where crude was removed so as not to skew the magnitude of the scale.
This helped me see some interesting patterns in gas (green) and oil (purple), particularly their mutual rise and fall during the second half of the timeframe. I isolated these two products to look more closely at their interaction.
I noted the drop off of imports around 2008, and would be interested to investigate further to see if this could be recession related (perhaps by comparing other OECD countries during this time period).
In order to continue moving through my dataset, I switched my attention to comparing US imports with exports. When it comes to energy products, imports only tell half of the story, and I was curious to see if the US showed net deficits or net gains.
I chose to focus my attention on oil.Oil in this context includes refinery gas, ethane, aviation gasoline, motor gasoline, jet fuels, kerosene, gas/disel oil, fuel oil, and other oil products.
I could immediately see two distinct periods of time: 1) when imports were greater than exports, and 2) when exports were greater than imports. I shaded these in to see them more clearly.
I then calculated and plotted the total (ktoe) of the observed net gain.
I was intrigued by the parallel rise of imports and exports between around 1995 to 2006, so I calculated and plotted their correlation for this period.
export<- subset(
df.imports_exports_USA$Exports..ktoe.,
df.imports_exports_USA$Year < as.Date('2006', '%Y') &
df.imports_exports_USA$Year > as.Date('1995', '%Y')
)
import <- subset(
df.imports_exports_USA$Imports..ktoe.,
df.imports_exports_USA$Year < as.Date('2006', '%Y') &
df.imports_exports_USA$Year > as.Date('1995', '%Y')
)
export <- abs(export)
lm(export~import)
##
## Call:
## lm(formula = export ~ import)
##
## Coefficients:
## (Intercept) import
## 2.745e+04 2.188e-01
Their low correlation coeffecient led me to believe that one could not be used as a suitable predictor of the other for this period.
I shifted my attention back to crude, and plotted US crude imports against US crude exports.
The scale of the US crude net deficit was quite suprising, with exports only scarecly registering in the late 1990s and after around 2008. Also, as viewed initially in my import comparison, the magnitude of crude imports was far greater than what I had observed in oil. To help me visualize these relative magnitudes I plotted my US oil imports and exports against a backrdop of US crude imports and exports.
My US analysis led me to wonder how US energy compared to world totals. I started by plotting world energy imports.
Oil led the pack as the leading world energy import, so I singled it for closer examination.
World oil imports see a steady rise over time. I wanted to see if this was also true for exports, so I plotted them side by side.
export <- df.imports_exports_world$Exports..ktoe.
import <- df.imports_exports_world$Imports..ktoe.
lm(import~export)
##
## Call:
## lm(formula = import ~ export)
##
## Coefficients:
## (Intercept) export
## -3.291e+04 -9.631e-01
Unlike the import-export disarpity I observed in US crude, world oil imports and exports seem to advance in lockstep. Their correlation coefficient was incredibly strong (-0.96).
I was curiou about the total net gain acculmulated over time, so I calculated and plotted that directly.
netgain <- sum(abs(df.imports_exports_world$Exports..ktoe.) -
df.imports_exports_world$Imports..ktoe.)
netgain
## [1] 2586108
With over 2.5 Million ktoe in oil net gains, world totals paint a much fuller picture of the total oil market. To help me visualize this scale, I plotted US oil imports next to World oil imports.
In addition to data on specific OECD countries, this dataset includes regional data from non-OECD nations.
For comparison, I plotted OECD imports and exports alongside the non-OECD regions.
From these barcharts it was difficult to determine which country experienced the greatest oil net gain, so I calculated and plotted this directly as a pie chart.
Plotted in the pie cart below are the percentages of total positive regional net flow for oil from 1971-2014. These were calculated by subtracting crude imports from crude exports for each of the listed regions. The percentage listed for each region is that region’s total crude net flow for all years divided by the total positive crude net gain for all regions for all years (their slice of the regional net crude gain pie).
I removed Asia and OECD from the pie chart, as these two regions showed negative net oil flow.
As I expected, the Middle East is in the lead with 44% of all net positive oil flow across regions. I was curious to see if this was also true for crude.
For crude, the Middle East leads by an even bigger margin, with Africa, the second greatest percentage, coming in at a third of the Middle East’s percentage. Once again, Asia and OECD were ommitted from this barchart, as each had negative net crude flow.
Given the negative net crude for US and OECD, I wanted to see the percentage of negative OECD crude flow that could be accounted for by the US crude net deficit.
# Create OECD dataframe
df.ie_crude_OECD <- subset(df.ie_crude, Country == 'OECD Total')
# Plot OECD crude flow
ggplot(df.ie_crude_OECD, aes(x = Year)) +
geom_line(aes(y = abs(Exports..ktoe.), color = 'Export')) +
geom_line(aes(y = Imports..ktoe., color = 'Import')) +
ylab('OECD crude (ktoe)') +
geom_ribbon(
data=subset(df.ie_crude_OECD, Year > as.Date('1970', '%Y')),
aes(x = Year, ymin=abs(Exports..ktoe.),ymax=Imports..ktoe.),
fill="red",
alpha="0.5") +
scale_y_continuous(limits = c(0, 1800000), breaks = pretty_breaks(10)) +
annotate(
"text",
x = as.Date('2004', '%Y'),
y = 1000000,
label = "Net deficit")
# Calculate US percentage of OECD crude net loss
US_crude <- sum(df.ie_crude_USA$Imports..ktoe. -
df.ie_crude_USA$Exports..ktoe.)
OECD_crude <- sum(df.ie_crude_OECD$Imports..ktoe. -
df.ie_crude_OECD$Exports..ktoe.)
US_crude/OECD_crude
## [1] 0.2412082
While there are 34 countries in the OECD, the total US crude net deficit makes up 24% percent of the OECD total crude net deficit.
I began my analysis by examining US energy products, and quickly shifted my attention to oil. By plotting oil imports and exports together I was able to see that there was a net deficit between 1971 and 2009, which then shifted to a net gain as exports began to exceed imports. This trend increases over the last few years of the dataset.
Having seen how large US crude imports were in my initial analysis, I plotted these against exports and included a shaded depiction of the US crude net deificit. For comparison, I plotted this against a backdrop of the same relationship (crude imports vs. exports) for the entire OECD, and calculated the US percentage of this total crude net deficit (24%).
The IEA, the agency responsible for this datasest, is an idependent agency that works in close partnership with the OECD. Given this, it should come as no surprise that this dataset has extensive energy flow data for OECD partners, yet only overview regional data for non-OECD nations and regions.
That being said, even if we look just at regional crude oil net gains, we can begin to see outlines of the energy flow landscape of nations who export more crude than then import.
## [1] 11089208 -7214791 32822605 2496360 5946392 -47322953
This analysis began with an overview of US energy imports, shifted to a deeper look at US oil and crude, drew comparisons to world crude flow, and ended with a look at regional net flow. This trajectory was inspired by initial observations in my first plot that showed oil and crude to be leading US imports. This analysis was also motivated by my desire to have a fuller picture of how imports related to exports over time, as such an analysis begins to paint a part of the US energy economic picture for these products.
While there were complete sets of datapoints for oil and crude, I was dissapointed to find little to know data for some of the other US products, particularly for Renewables. It was not clear from the documentation whether this lack of data is due to low US participaion in the renewables market or a biproduct of the data collection process. A deeper study of the IEA’s data collection methods would be required to answer this question, and would be advised before providing any analysis of these energy products with sparse or no data.
While I enjoyed the ktoe accuracy in these data, I found myself wishing to plot net deficits and gains in terms of costs (perhaps in USD or GBP). This would require cross-referencing energy trading prices at the time of data collection. Such analysis would help provide a better understanding of specific economic realities of these product types. This includes industry applications, trade restrictions, and efficiency as a fuel source. Such information, combined with cost, could help transform the data here into a more unified picture of global energy value distribution.